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Abstract 


Based on Space Station and planetary spacecraft communication time delays and 
bandwidth limitations, it will be necessary to develop an intelligent, general purpose ground 
monitor terminal capable of sophisticated data display and control of on-orbit facilities and 
remote spacecraft. The basic elements that make up a Monitor and Simulation Terminal 
(MASTER) include computer overlay video, data compression, forward simulation, mission 
resource optimization and high level robotic control. Hardware and software elements of a 
MASTER are being assembled for testbed use. 


Applications of Neural Networks (NNs) to some key functions of a MASTER are also 
discussed. These functions are overlay graphics adjustment, object correlation and kinematic- 
dynamic characterization of the manipulator. 


1.0 Introduction 

Teledyne Brown Engineering (TBE) is completing the independent research and 
development of a Monitor and Simulation Terminal (MASTER) for use as a Space Station 
telescience terminal. This effort complements the TBE responsibility for outfitting the Space 
Station U.S. Laboratory. The terminal is being used within the Robotics Laboratory to 
investigate techniques to enhance space experiment platform resources such as: 


1) Communications bandwidth 

2) Crewtime 

3) Microgravity 
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The MASTER is a key element in this effort because it provides the ability to conduct 
telescience prior to flight and also execute later remote operation of actual equipment and robot 
systems. The functions that will typically reside in the MASTER are: 

1) Video Processing and Overlay 

2) Predictive Simulation 

3) Intuitive User Interfaces 

4) Decision Assistance 

5) 3D Modeling and Graphics 

The goal of this effort is to provide a telescience platform which can be tailored to 
specific experimenter resource and operational requirements. The results of this activity will 
allow a better understanding of user requirements and improve the design of teleoperated 
experiments. 

2.0 Monitor and Simulation Requirements 

Predictive Simulation requirements for a MASTER system were first alluded to by 
Ferrell in 1965 [1], More recently, Akin [2] and Konkel [3] et al have discussed the importance 
of predictive visual and force reflective simulation. The simulation and 3D modeling features of 
the testbed are based on these requirements. Initially our effort has been limited to a predictive 
visual simulation due to the complexities of predictive force reflection. Our design is also driven 
by user requirements such as: 

1) Maximizing the amount of useful information while minimizing on-screen 
"clutter" 

2) Minimizing head motion to different displays. 

3) Providing a mode for uplink control of the manipulator during special situations 
or contingencies. 

3.0 System Design 

Basic design requirements for a MASTER are shown in Figure 1. Primary elements 
include the Simulation Platform and Monitor. The Simulation Platform communicates with the 
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remote site, which includes both robotic and automation elements, as shown in the upper 
portion of Figure 1. 

A simulation-based testbed was chosen for validating algorithms and state of the art 
control concepts. In this manner, new teleoperations and remote monitoring methods can be 
easily integrated and adjusted. The task of integrating graphics terminals, video cameras, 
neurocomputers, and robot manipulators, however, is non-trivial. Although each hardware 
component is capable of individually meeting its own functional requirements, the combined 
system must be properly integrated. 


The TBE Robotics Laboratory provides a simulated remote work site. This will allow 
MASTER to ultimately become part of an actual integrated telescience testbed. This includes 
hardware in-the-loop to simulate time-delayed teleoperations. 

A great deal of attention has therefore gone into the design of a "friendly" interface. The 
operator can utilize a range of input devices. Mouse, keyboard, voice and miniature master 
manipulators have been used as control inputs. To operate effectively in the presence of time 
delay, all user inputs can be processed by the predictive simulation. The path of the robot can 
be modeled by the simulation, in real time, based on the user’s inputs. An attempt has been 
made to minimize the number of displays required. Ideally, a single high-resolution terminal 
with multiple (possibly voice controlled) overlay windows would suffice. Dual orthogonal 
camera views using windows were critical during robot control. MASTER also provides an 
optimum human-machine interface such that an untrained tele-experimenter can become a 
competent tele-operator within a short time. 

The functional design of the MASTER Simulation Platform is shown in the block 
diagram of Figure 2. In order to have a useful predictive simulation an accurate dynamic and 
kinematic model of the remote manipulator is provided. In addition, the 3-dimensional 
perspective of the simulation is automatically aligned with live video from the remote location. 
The graphics engine, world model, math models (i.e., kinematic/dynamic manipulator models), 
and neural network models are all used by the predictive simulation. 

A high-resolution graphics workstation, with its own geometric pipeline processor, 
forms the heart of the Simulation Platform. The graphics engine, math models and world 
models all reside within this computer. Video overlay is accomplished by simulating both the 
"live" and the predictive views or by the use of live video data and a frame grabber-mixer. The 
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neural netwoik models reside on a PC based neurocomputer workstation. The neurocomputer 
hosts a scientific vector processing board and associated neural netwoik simulation software. 

4.0 Work in Progress 

This section briefly discusses our current work in progress. We are emphasizing neural 
computing applications for use in the MASTER. 

4. 1 Neurocomputing 

Neurocomputing methods and hardware have been applied to the predictive display 
system. Two such problems are perspective adjustment for accurate simulation overlay and 
adaptive modeling of the manipulator dynamics. Other areas under consideration are two- 
dimensional and three-dimensional image tracking and compression, limited resource allocation 
and force-eye-hand coordination. 

We have demonstrated an overlay perspective calibration technique that uses NNs. The 
problem is to align the perspective of two superposed images so that they appear to be the same 
image. Figure 3 schematically illustrates the problem we are addressing and the neural 
controller. Given visual data from an on-orbit (remote) site and a simulated view of that site, 
control of the simulated perspective until the images overlap is required. The method 
demonstrated uses a vision computer to digitize each image separately. These images are then 
processed to provide a measure of translational and scale offsets between the images. These 
offsets along with the control commands that an operator would use to manually align the 
images are then given to a NN that uses the generalized delta learning rule of Rummelhart [4] 
(know as back propagation) to ’Team" how to align the images as an operator would. We have 
been successful in training our system to do perspective line-up to within 5% overlap. A simple 
logic algorithm has also been developed that will perform this same task. On comparing this 
algorithm to the neural architectures, it was found that the logic algorithm can be considered a 
computational subset of the neural network. Figure 4 illustrates the three perspective calibration 
techniques investigated. 

Dynamics modeling is possibly the most difficult problem faced by a predictive 
simulation. The low gravitational environment in space telerobotics may significantly alter the 
dynamic response of manipulators. If the manipulator dynamic response is not properly 
compensated for in the predictive model the simulation may lose its effectiveness. TBE has 
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begun experimenting with NN modeling of the kinematics and dynamics of two-dimensional 
manipulators. Application of Kohonen's neural model for vector quantization and self- 
organization [5] has yielded encouraging results. Figure 5 shows how a Kohonen NN can 
represent the kinematic workspace. The web in Figure 5 is generated by plotting the actual 
synaptic coupling weights (two per neuron) as two-dimensional points. Lines are then drawn to 
connect topologically nearest-neighbor neurons. Initially the weights are random; thus the net 
appears tangled. During learning, random joint angles are used to reposition the arm for each 
cycle. As learning proceeds the network weights form a representation of the kinematic 
workspace. Once the neural workspace is created, end effector position vectors along a 
trajectory can be used to stimulate the trained neurons. This will result in unique time-varying 
neural activation patterns for every possible trajectory. Figure 6 gives a qualitative view of a 
typical activity pattern over time. The amount of nodal stimulation will depend on many factors 
such as trajectory path, speed, acceleration, and neural time constants. We feel that this type of 
kinematic/dynamic modeling may be useful in the analysis of dynamic changes that can occur 
under different working environments. 

To date, other internal research at TBE has developed a method for constructing practical 
optical processing components that will eventually provide a MASTER with very high-speed, 
cost-effective neural network computing power. These include a fixed interconnect optical 
neural network, a small ragged optical correlation device, and optical associative memory. 

5.0 Summary and Conclusions 

Efficient man-machine telescience is the goal of this effort. In order to meet such a goal 
the problem must be viewed from the end user’s point of view. We have concluded that a good 
man-machine interface for practical telescience will require a sophisticated predictive simulation 
and monitoring platform that is transparent to the user and second nature to operate. This will 
require the application of new machine intelligence technologies, particularly neural computing. 
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FIGURE 1. BASIC REQUIREMENTS FOR A MASTER SYSTEM AND REMOTE WORK SITE 
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INTERNAL VIEW: SIMULATION PLATFORM 
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EXTERNAL VIEW: MASTER 
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FIGURE 2. SIMULATION PLATFORM BASIC REQUIREMENTS 
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VISION DATA FROM ORBIT SYSTEM 



CONTROL COMMANDS | 

890209-002M3-1 1 


i i 

VISION COMPUTER 


J 



FIGURE 3. INTERNAL RESEARCH DEVELOPMENT OF NEURAL 
TECHNIQUES FOR PERSPECTIVE ADJUSTMENT 
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FIGURE 5. SELF ORGANIZATION FOR MAPPING ARM WORKSPACE/ 

PROBLEM DOMAIN 
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FIGURE 6. ACTIVATION PATTERNS IN SELF-ORGANIZED MAPPINGS: 

DYNAMICS MODELING 
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